It is known in the entertainment industry to use realistic computer graphics (CG) in various aspects of movie production. Many algorithms for natural behavior in the visual domain have been developed for film. For example, algorithms were developed for movies such as Jurassic Park to determine how a natural gait looked, how muscles moved in relation to a skeleton and how light reflected off of skin. However, similar types of problems in the audio, particularly music, domain remain relatively unaddressed. The necessary step is the ability to accurately transcribe what happens in a music performance into precise measurements that allow the fine nuances of the performance to be recreated.
Characterizing music may be a particularly difficult problem. Various approaches have been attempted to providing “automatic transcription” of music, typically from a waveform audio (WAV) format to a Musical Instrument Digital Interface (MIDI) format. Computer musicians generally refer to “WAV-to-MIDI” with reference to transforming a song in digitized waveforms into the corresponding notes in the MIDI format. The source of the recording could be, for example, analog or digital, and the conversion process can start from a record, tape, CD, MP3 file, or the like. Traditional musicians generally refer to such transformation of a song as “Automatic Transcription.” Manual transcription techniques are typically used by skilled musicians who listen to recordings repeatedly and carefully copy down on a music score the notes they hear; for example, to notate improvised jazz performances.
Numerous academics have looked at some of the problems in a non-commercial context. In addition, various companies offer software for WAV-to-MIDI decoding, for example, Digital Ear™, intelliScore™, Amazing MIDI, AKoff™, MB TRANS™, and Transcribe!™. These products generally focus on songwriters and amateurs and include capability for determining note pitches and durations, to help musicians create a simple score from a recording. However, these known products tend to be generally unreliable in processing more than one note at a time. In addition, these products generally fail to address the full range of characteristics of music. For example, with a piano, note characteristics may include: pitch, duration, strike and release velocities, key angle, and pedals. Academic research on automatic transcription has also occurred, for example, at the Tampere University of Technology in Finland. Known work on automatic transcription has generally not yielded archival-quality recreation of music performances.
There are 100 years of recordings in the vaults of the recording companies and in private collections. Many great recordings have never been released, because they were marred in some way that made them substandard. Live performances are often commercially not releaseable because of background noises or out-of-tune piano strings. Many analog tapes from previous decades are decaying, because of the chemical formula used in making the tape binder. They also may never have been released because they were recorded on low-quality devices, such as cassette recorders. Similarly, many desirable studio recordings have never seen released, due to instrument or equipment problems during their recording sessions.
The recording industry has embarked on the next set of consumer formats, following CDs in the early 1980's: high-definition surround sound. The new formats include DVD-Audio (DVD-A) and Video and Super Audio CD (SACD). There are 33 million home surround sound systems in use today, a number growing quickly along with high-definition TV. The challenge in the recording industry is bringing older audio material forward into modern sound for re-release. Candidates for such a conversion include mono recordings, especially those before 1955; stereo recordings without multi-channel masters; master tapes from the 1970s and 1980s, which are generally now decaying due to an inferior tape binder formulation; and any of these combined with video captures, which are issued as surround-sound DVDs.
Another music related recording area is creating MIDI from a printed score. For example, like optical character reader (OCR) software for text documents, it is known to provide application software for musicians to allow them to place a music score on a scanner and have music-scan application software convert it into a digitized format based on the scanned image. Similarly, application notation software is known to convert MIDI files to printed musical scores.
Application software for converting from MIDI to WAV is also known. The media player on a personal computer typically plays MIDI files. The better the samples it uses (snippets of digital recordings of acoustic instruments), the better the playback will typically sound. MIDI was originally designed, at least in part, as a way to describe performance details to electronic musical instruments, such as MIDI electronic pianos (with no strings or hammers) available, for example, from Korg, Kurzweil, Roland, and Yamaha.